The heterogeneity of applications and their divergent resource requirements lead to uneven traffic distribution and imbalanced resource utilization across data center networks (DCNs). We propose a fine-grained baseband function reallocation scheme in heterogeneous optical switching-based DCNs. A deep reinforcement learning-based functional split and resource mapping approach (DRL-BFM) is proposed to maximize throughput in high-load server racks by implementing load balancing in DCNs. The results demonstrate that DRL-BFM improves the throughput by 20.8%, 22.8%, and 29.8% on average compared to existing algorithms under different computational capacities, bandwidth constraints, and latency conditions, respectively.